Two Repeated Measures Experiments (2022)
An art-led research project developed during Convention House, an art research residency funded by the Garfield Weston Foundation via East Street Arts
Convention House, East Street Arts, Leeds, July-November 2022
Armelle Skatulski, February 2023
Drawing study for “Two Repeated-Measures Experiments” - an art project researching gestural motion data collection in VR © 2022 Armelle Skatulski.
Convention House // East Street Arts // Weston Culture Fund
I was recently commissioned to undertake an artist residency at Convention House, East Street Arts (Leeds, UK) supported by the Weston Culture Fund (July-November 2022). I started the residency by researching avenues for community-led Virtual Reality platforms as opposed to platforms owned by large corporations or technology monopolies.
My desire to do so was informed by my previous experience as a commissioned artist/researcher working for progressive think tanks with a focus on the personal data economy, the datafication of work and digital rights advocacy - The Autonomy Institute (London) and The UWA Tech & Policy Lab (University of Western Australia, Perth). What motivated me was to find ways to use artistic practice to translate knowledge usually disseminated through academic papers or policy reports, so as to reach a wider public.
Image: from an Oculus Rift patent, Facebook Reality Labs, U.S. Patent No. 9,063,330, USPTO, 2019.
Mapping Virtual Presence in Metaverse Infrastructures
I wanted to find tangible or practical ways to show what happens behind the scenes of VR experience so to speak: in terms of data collection, the ownership of networked infrastructures, and in terms of forms of labour involved. Hence, it was about finding a kind of material and formal prism through which I could excavate the various strata of relations existent between people and corporate entities, as mediated by XR technologies.
Image: Armelle Skatulski, Mapping Metaverse Infrastructures (diagram section), 2022.
Diagram © 2022 Armelle Skatulski All Rights Reserved.
Speculative Coding as Artistic Methodology Informing Interaction Design
I came across some exciting projects of collective, community-based ownership of communication infrastructures and of digital justice initiatives in the US (such as the Detroit Community Technology Project). I paused that avenue of my research to turn my attention to the practicalities of VR development (using game engines, e.g. Unity3D) and to the mechanisms used by technology companies to record and map all sorts of information from people during VR interactions.
As for instance the type of video or image capture enabled by VR head mounted devices (HMD) used to map and predict facial expressions or motion. There is speculation as to whether images captured via HMDs may be used to train algorithms, produce behavioural statistics and extract value from users in VR.
I researched which practical paths could be available to me to speculatively represent the black-boxed mechanisms of data collection used by large platforms such as Meta LLC to track people’s activity and collect personal data during VR interactions.
The answer was: CODE and LOGIC. Or rather speculative coding.
I decided to design a simple application using the game engine Unity3D which would contain code that would collect various forms of information or data from different types of gestural motion. I would then transpose these to a form of 3D representation through simple data visualisation tools or by translating them as editable, VR-ready 3D geometry with appropriate software.
Designing “Two Repeated-Measures Experiments” (2022)
I designed and produced two gaming applications with technical support from a developer. These were inspired by scientific experiments on ergonomic risk which were undertaken by Facebook’s Reality Labs.
The C# scripts written by Stuart Mellor and contained in the infrastructure of my ‘games’ is aimed at recording and mapping different types of gestural motion during game usage. The code uses the CommonUsages API available for developers on Unity3D’s open source repository.
This led me to present “Two repeated-measures experiments” (2022-ongoing) to a group of designers and artists during a public workshop at Convention House in November 2022. Participants tested two VR tasks (one of which a painting task, the other a randomised target exercise) and were shown how distinctive gestural movements could be captured with code and then translated through 3D plotting and visualisation tools or expressed formally through artistic strategies.
Image: Drawing study for "A Randomised Target" from “Two Repeated-Measures Experiments” (2022)
Image: Mapping Metaverse Infrastructures (diagram section)
Diagram © 2022 - Armelle Skatulski All Rights Reserved.
"Two Repeated Measures Experiments" being tested by members of the public during a free workshop, Convention House studios, East Street Arts, Leeds, November 2022.
